<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 2.0.22">
<title>Strimzi Overview</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700">
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.18.3/styles/github.min.css">
</head>
<body id="overview-book_str" class="book toc2 toc-left">
<div id="header">
<h1>Strimzi Overview</h1>
<div id="toc" class="toc2">
<div id="toctitle">Table of Contents</div>
<ul class="sectlevel1">
<li><a href="#key-features_str">1. Key features</a>
<ul class="sectlevel2">
<li><a href="#key-features-kafka_str">1.1. Kafka capabilities</a></li>
<li><a href="#kafka_use_cases">1.2. Kafka use cases</a></li>
<li><a href="#key-features-product_str">1.3. How Strimzi supports Kafka</a></li>
</ul>
</li>
<li><a href="#kafka-components_str">2. Strimzi deployment of Kafka</a>
<ul class="sectlevel2">
<li><a href="#kafka-concepts-components_str">2.1. Kafka component architecture</a></li>
</ul>
</li>
<li><a href="#kafka-concepts_str">3. About Kafka</a>
<ul class="sectlevel2">
<li><a href="#kafka-concepts-key_str">3.1. How Kafka operates as a message broker</a></li>
<li><a href="#kafka-concepts-producers-consumers_str">3.2. Producers and consumers</a></li>
</ul>
</li>
<li><a href="#kafka-connect-components_str">4. About Kafka Connect</a>
<ul class="sectlevel2">
<li><a href="#key-features-kafka-connect_str">4.1. How Kafka Connect streams data</a>
<ul class="sectlevel3">
<li><a href="#connectors">4.1.1. Connectors</a></li>
<li><a href="#tasks">4.1.2. Tasks</a></li>
<li><a href="#workers">4.1.3. Workers</a></li>
<li><a href="#transforms">4.1.4. Transforms</a></li>
<li><a href="#converters">4.1.5. Converters</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#overview-components-kafka-bridge_str">5. Kafka Bridge interface</a>
<ul class="sectlevel2">
<li><a href="#http_requests">5.1. HTTP requests</a></li>
<li><a href="#con-overview-components-kafka-bridge-clients_str">5.2. Supported clients for the Kafka Bridge</a></li>
</ul>
</li>
<li><a href="#overview-components_str">6. Strimzi Operators</a>
<ul class="sectlevel2">
<li><a href="#overview-components-cluster-operator-str">6.1. Cluster Operator</a></li>
<li><a href="#overview-concepts-topic-operator-str">6.2. Topic Operator</a></li>
<li><a href="#overview-concepts-user-operator-str">6.3. User Operator</a></li>
<li><a href="#con-feature-gates-overview-str">6.4. Feature gates in Strimzi Operators</a></li>
</ul>
</li>
<li><a href="#configuration-points_str">7. Kafka configuration</a>
<ul class="sectlevel2">
<li><a href="#configuration-points-resources_str">7.1. Custom resources</a></li>
<li><a href="#configuration-points-common_str">7.2. Common configuration</a></li>
<li><a href="#configuration-points-broker_str">7.3. Kafka cluster configuration</a></li>
<li><a href="#configuration-points-node_pools_str">7.4. Kafka node pools configuration</a></li>
<li><a href="#con-configuration-points-mm2-str">7.5. Kafka MirrorMaker 2 configuration</a></li>
<li><a href="#con-configuration-points-mm-str">7.6. Kafka MirrorMaker configuration</a></li>
<li><a href="#configuration-points-connect_str">7.7. Kafka Connect configuration</a></li>
<li><a href="#configuration-points-bridge_str">7.8. Kafka Bridge configuration</a></li>
</ul>
</li>
<li><a href="#security-overview_str">8. Securing Kafka</a>
<ul class="sectlevel2">
<li><a href="#security-configuration-encryption_str">8.1. Encryption</a></li>
<li><a href="#security-configuration-authentication_str">8.2. Authentication</a></li>
<li><a href="#security-configuration-authorization_str">8.3. Authorization</a></li>
<li><a href="#security-configuration-fips_str">8.4. Federal Information Processing Standards (FIPS)</a></li>
</ul>
</li>
<li><a href="#metrics-overview_str">9. Monitoring</a>
<ul class="sectlevel2">
<li><a href="#metrics-overview-tools-str">9.1. Prometheus</a></li>
<li><a href="#metrics-overview-grafana_str">9.2. Grafana</a></li>
<li><a href="#metrics-overview-exporter_str">9.3. Kafka Exporter</a></li>
<li><a href="#metrics-overview-tracing_str">9.4. Distributed tracing</a></li>
<li><a href="#metrics-overview-cruisecontrol_str">9.5. Cruise Control</a></li>
</ul>
</li>
</ul>
</div>
</div>
<div id="content">
<div class="sect1">
<h2 id="key-features_str"><a class="link" href="#key-features_str">1. Key features</a></h2>
<div class="sectionbody">
<div class="paragraph _abstract">
<p>Strimzi simplifies the process of running <a href="https://kafka.apache.org/" target="_blank" rel="noopener">Apache Kafka</a> in a Kubernetes cluster.</p>
</div>
<div class="paragraph">
<p>This guide is intended as a starting point for building an understanding of Strimzi.
The guide introduces some of the key concepts behind Kafka, which is central to Strimzi, explaining briefly the purpose of Kafka components.
Configuration points are outlined, including options to secure and monitor Kafka.
A distribution of Strimzi provides the files to deploy and manage a Kafka cluster, as well as <a href="./deploying.html#config-examples-str" target="_blank" rel="noopener">example files for configuration and monitoring of your deployment</a>.</p>
</div>
<div class="paragraph">
<p>A typical Kafka deployment is described, as well as the tools used to deploy and manage Kafka.</p>
</div>
<div class="sect2">
<h3 id="key-features-kafka_str"><a class="link" href="#key-features-kafka_str">1.1. Kafka capabilities</a></h3>
<div class="paragraph">
<p>The underlying data stream-processing capabilities and component architecture of Kafka can deliver:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Microservices and other applications to share data with extremely high throughput and low latency</p>
</li>
<li>
<p>Message ordering guarantees</p>
</li>
<li>
<p>Message rewind/replay from data storage to reconstruct an application state</p>
</li>
<li>
<p>Message compaction to remove old records when using a key-value log</p>
</li>
<li>
<p>Horizontal scalability in a cluster configuration</p>
</li>
<li>
<p>Replication of data to control fault tolerance</p>
</li>
<li>
<p>Retention of high volumes of data for immediate access</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="kafka_use_cases"><a class="link" href="#kafka_use_cases">1.2. Kafka use cases</a></h3>
<div class="paragraph">
<p>Kafka&#8217;s capabilities make it suitable for:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Event-driven architectures</p>
</li>
<li>
<p>Event sourcing to capture changes to the state of an application as a log of events</p>
</li>
<li>
<p>Message brokering</p>
</li>
<li>
<p>Website activity tracking</p>
</li>
<li>
<p>Operational monitoring through metrics</p>
</li>
<li>
<p>Log collection and aggregation</p>
</li>
<li>
<p>Commit logs for distributed systems</p>
</li>
<li>
<p>Stream processing so that applications can respond to data in real time</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="key-features-product_str"><a class="link" href="#key-features-product_str">1.3. How Strimzi supports Kafka</a></h3>
<div class="paragraph">
<p>Strimzi provides container images and operators for running Kafka on Kubernetes.
Strimzi operators are purpose-built with specialist operational knowledge to effectively manage Kafka on Kubernetes.</p>
</div>
<div class="paragraph">
<p>Operators simplify the process of:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Deploying and running Kafka clusters</p>
</li>
<li>
<p>Deploying and running Kafka components</p>
</li>
<li>
<p>Configuring access to Kafka</p>
</li>
<li>
<p>Securing access to Kafka</p>
</li>
<li>
<p>Upgrading Kafka</p>
</li>
<li>
<p>Managing brokers</p>
</li>
<li>
<p>Creating and managing topics</p>
</li>
<li>
<p>Creating and managing users</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="kafka-components_str"><a class="link" href="#kafka-components_str">2. Strimzi deployment of Kafka</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Apache Kafka components are provided for deployment to Kubernetes with the Strimzi distribution.
The Kafka components are generally run as clusters for availability.</p>
</div>
<div class="paragraph">
<p>A typical deployment incorporating Kafka components might include:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><strong>Kafka</strong> cluster of broker nodes</p>
</li>
<li>
<p><strong>ZooKeeper</strong> cluster of replicated ZooKeeper instances</p>
</li>
<li>
<p><strong>Kafka Connect</strong> cluster for external data connections</p>
</li>
<li>
<p><strong>Kafka MirrorMaker</strong> cluster to mirror the Kafka cluster in a secondary cluster</p>
</li>
<li>
<p><strong>Kafka Exporter</strong> to extract additional Kafka metrics data for monitoring</p>
</li>
<li>
<p><strong>Kafka Bridge</strong> to make HTTP-based requests to the Kafka cluster</p>
</li>
<li>
<p><strong>Cruise Control</strong> to rebalance topic partitions across broker nodes</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Not all of these components are mandatory, though you need Kafka and ZooKeeper as a minimum.
Some components can be deployed without Kafka, such as MirrorMaker or Kafka Connect.</p>
</div>
<div class="sect2">
<h3 id="kafka-concepts-components_str"><a class="link" href="#kafka-concepts-components_str">2.1. Kafka component architecture</a></h3>
<div class="paragraph">
<p>A cluster of Kafka brokers handles delivery of messages.</p>
</div>
<div class="paragraph">
<p>A broker uses Apache ZooKeeper for storing configuration data and for cluster coordination.
Before running Apache Kafka, an Apache ZooKeeper cluster has to be ready.</p>
</div>
<div class="paragraph">
<p>Each of the other Kafka components interact with the Kafka cluster to perform specific roles.</p>
</div>
<div class="paragraph">
<div class="title">Kafka component interaction</div>
<p><span class="image"><img src="images/overview/kafka-concepts-supporting-components.png" alt="Data flows between several Kafka components and the Kafka cluster. See the component descriptions after this image."></span></p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Apache ZooKeeper</dt>
<dd>
<p>Apache ZooKeeper is a core dependency for Kafka as it provides a cluster coordination service, storing and tracking the status of brokers and consumers. ZooKeeper is also used for controller election.</p>
</dd>
<dt class="hdlist1">Kafka Connect</dt>
<dd>
<p>Kafka Connect is an integration toolkit for streaming data between Kafka brokers and other systems using <em>Connector</em> plugins.
Kafka Connect provides a framework for integrating Kafka with an external data source or target, such as a database, for import or export of data using connectors.
Connectors are plugins that provide the connection configuration needed.</p>
<div class="ulist">
<ul>
<li>
<p>A <em>source</em> connector pushes external data into Kafka.</p>
</li>
<li>
<p>A <em>sink</em> connector extracts data  out of Kafka</p>
<div class="paragraph">
<p>External data is translated and transformed into the appropriate format.</p>
</div>
<div class="paragraph">
<p>You can deploy Kafka Connect with <code>build</code> configuration that automatically builds a container image with the connector plugins you require for your data connections.</p>
</div>
</li>
</ul>
</div>
</dd>
<dt class="hdlist1">Kafka MirrorMaker</dt>
<dd>
<p>Kafka MirrorMaker replicates data between two Kafka clusters, within or across data centers.</p>
<div class="paragraph">
<p>MirrorMaker takes messages from a source Kafka cluster and writes them to a target Kafka cluster.</p>
</div>
</dd>
<dt class="hdlist1">Kafka Bridge</dt>
<dd>
<p>Kafka Bridge provides an API for integrating HTTP-based clients with a Kafka cluster.</p>
</dd>
<dt class="hdlist1">Kafka Exporter</dt>
<dd>
<p>Kafka Exporter extracts data for analysis as Prometheus metrics, primarily data relating to offsets, consumer groups, consumer lag and topics. Consumer lag is the delay between the last message written to a partition and the message currently being picked up from that partition by a consumer</p>
</dd>
</dl>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="kafka-concepts_str"><a class="link" href="#kafka-concepts_str">3. About Kafka</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Apache Kafka is an open-source distributed publish-subscribe messaging system for fault-tolerant real-time data feeds.</p>
</div>
<div class="paragraph">
<p>For more information about Apache Kafka, see the <a href="https://kafka.apache.org/documentation/" target="_blank" rel="noopener">Apache Kafka documentation</a>.</p>
</div>
<div class="sect2">
<h3 id="kafka-concepts-key_str"><a class="link" href="#kafka-concepts-key_str">3.1. How Kafka operates as a message broker</a></h3>
<div class="paragraph _abstract">
<p>To maximise your experience of using Strimzi, you need to understand how Kafka operates as a message broker.</p>
</div>
<div class="paragraph">
<p>A Kafka cluster comprises multiple brokers.
Brokers contain topics that receive and store data.
Topics are split by partitions, where the data is written.
Partitions are replicated across topics for fault tolerance.</p>
</div>
<div class="paragraph">
<div class="title">Kafka brokers and topics</div>
<p><span class="image"><img src="images/overview/kafka-concepts-key-concepts.png" alt="Kafka brokers and topics inside a Kafka cluster showing the partition leader of each topic"></span></p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Broker</dt>
<dd>
<p>A broker, sometimes referred to as a server or node, orchestrates the storage and passing of messages.</p>
</dd>
<dt class="hdlist1">Topic</dt>
<dd>
<p>A topic provides a destination for the storage of data.
Each topic is split into one or more partitions.</p>
</dd>
<dt class="hdlist1">Cluster</dt>
<dd>
<p>A group of broker instances.</p>
</dd>
<dt class="hdlist1">Partition</dt>
<dd>
<p>The number of topic partitions is defined by a topic <em>partition count</em>.</p>
</dd>
<dt class="hdlist1">Partition leader</dt>
<dd>
<p>A partition leader handles all producer requests for a topic.</p>
</dd>
<dt class="hdlist1">Partition follower</dt>
<dd>
<p>A partition follower replicates the partition data of a partition leader, optionally handling consumer requests.</p>
<div class="paragraph">
<p>Topics use a <em>replication factor</em> to configure the number of replicas of each partition within the cluster.
A topic comprises at least one partition.</p>
</div>
<div class="paragraph">
<p>An <em>in-sync</em> replica has the same number of messages as the leader.
Configuration defines how many replicas must be <em>in-sync</em> to be able to produce messages, ensuring that a message is committed only after it has been successfully copied to the replica partition.
In this way, if the leader fails the message is not lost.</p>
</div>
<div class="paragraph">
<p>In the <em>Kafka brokers and topics</em> diagram, we can see each numbered partition has a leader and two followers in replicated topics.</p>
</div>
</dd>
</dl>
</div>
</div>
<div class="sect2">
<h3 id="kafka-concepts-producers-consumers_str"><a class="link" href="#kafka-concepts-producers-consumers_str">3.2. Producers and consumers</a></h3>
<div class="paragraph">
<p>Producers and consumers send and receive messages (publish and subscribe) through brokers.
Messages comprise an optional <em>key</em> and a <em>value</em> that contains the message data, plus headers and related metadata.
The key is used to identify the subject of the message, or a property of the message.
Messages are delivered in batches, and batches and records contain headers and metadata that provide details that are useful for filtering and routing by clients, such as the timestamp and offset position for the record.</p>
</div>
<div class="paragraph">
<div class="title">Producers and consumers</div>
<p><span class="image"><img src="images/overview/kafka-concepts-producer-consumer.png" alt="A producer sends messages through a broker to a topic containing three partitions. Three consumers in a consumer group read the messages from the partitions"></span></p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Producer</dt>
<dd>
<p>A producer sends messages to a broker topic to be written to the end offset of a partition.
Messages are written to partitions by a producer on a round robin basis, or to a specific partition based on the message key.</p>
</dd>
<dt class="hdlist1">Consumer</dt>
<dd>
<p>A consumer subscribes to a topic and reads messages according to topic, partition and offset.</p>
</dd>
<dt class="hdlist1">Consumer group</dt>
<dd>
<p>Consumer groups are used to share a typically large data stream generated by multiple producers from a given topic.
Consumers are grouped using a <code>group.id</code>, allowing messages to be spread across the members.
Consumers within a group do not read data from the same partition, but can receive data from one or more partitions.</p>
</dd>
<dt class="hdlist1">Offsets</dt>
<dd>
<p>Offsets describe the position of messages within a partition.
Each message in a given partition has a unique offset, which helps identify the position of a consumer within the partition to track the number of records that have been consumed.</p>
<div class="paragraph">
<p>Committed offsets are written to an offset commit log.
A <code>__consumer_offsets</code> topic stores information on committed offsets, the position of last and next offset, according to consumer group.</p>
</div>
</dd>
</dl>
</div>
<div class="paragraph">
<div class="title">Producing and consuming data</div>
<p><span class="image"><img src="images/overview/kafka-concepts-partitions.png" alt="A producer sends a message to a broker topic; the message is written to the end offset (7). A consumer reads messages from offset 5"></span></p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="kafka-connect-components_str"><a class="link" href="#kafka-connect-components_str">4. About Kafka Connect</a></h2>
<div class="sectionbody">
<div class="paragraph _abstract">
<p>Kafka Connect is an integration toolkit for streaming data between Kafka brokers and other systems.
The other system is typically an external data source or target, such as a database.</p>
</div>
<div class="paragraph">
<p>Kafka Connect uses a plugin architecture to provide the implementation artifacts for connectors.
Plugins allow connections to other systems and provide additional configuration to manipulate data.
Plugins include connectors and other components, such as data converters and transforms.
A connector operates with a specific type of external system.
Each connector defines a schema for its configuration.
You supply the configuration to Kafka Connect to create a connector instance within Kafka Connect.
Connector instances then define a set of tasks for moving data between systems.</p>
</div>
<div class="paragraph">
<p>Strimzi operates Kafka Connect in <em>distributed mode</em>, distributing data streaming tasks across one or more worker pods.
A Kafka Connect cluster comprises a group of worker pods.
Each connector is instantiated on a single worker.
Each connector comprises one or more tasks that are distributed across the group of workers.
Distribution across workers permits highly scalable pipelines.</p>
</div>
<div class="paragraph">
<p>Workers convert data from one format into another format that&#8217;s suitable for the source or target system.
Depending on the configuration of the connector instance, workers might also apply transforms (also known as Single Message Transforms, or SMTs).
Transforms adjust messages, such as filtering certain data, before they are converted.
Kafka Connect has some built-in transforms, but other transformations can be provided by plugins if necessary.</p>
</div>
<div class="sect2">
<h3 id="key-features-kafka-connect_str"><a class="link" href="#key-features-kafka-connect_str">4.1. How Kafka Connect streams data</a></h3>
<div class="paragraph _abstract">
<p>Kafka Connect uses connector instances to integrate with other systems to stream data.</p>
</div>
<div class="paragraph">
<p>Kafka Connect loads existing connector instances on start up and distributes data streaming tasks and connector configuration across worker pods.
Workers run the tasks for the connector instances.
Each worker runs as a separate pod to make the Kafka Connect cluster more fault tolerant.
If there are more tasks than workers, workers are assigned multiple tasks.
If a worker fails, its tasks are automatically assigned to active workers in the Kafka Connect cluster.</p>
</div>
<div class="paragraph">
<p>The main Kafka Connect components used in streaming data are as follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Connectors to create tasks</p>
</li>
<li>
<p>Tasks to move data</p>
</li>
<li>
<p>Workers to run tasks</p>
</li>
<li>
<p>Transforms to manipulate data</p>
</li>
<li>
<p>Converters to convert data</p>
</li>
</ul>
</div>
<div class="sect3">
<h4 id="connectors"><a class="link" href="#connectors">4.1.1. Connectors</a></h4>
<div class="paragraph">
<p>Connectors can be one of the following type:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Source connectors that push data into Kafka</p>
</li>
<li>
<p>Sink connectors that extract data out of Kafka</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Plugins provide the implementation for Kafka Connect to run connector instances.
Connector instances create the tasks required to transfer data in and out of Kafka.
The Kafka Connect runtime orchestrates the tasks to split the work required between the worker pods.</p>
</div>
<div class="paragraph">
<p>MirrorMaker 2 also uses the Kafka Connect framework.
In this case, the external data system is another Kafka cluster.
Specialized connectors for MirrorMaker 2 manage data replication between source and target Kafka clusters.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<div class="title">Note</div>
</td>
<td class="content">
<div class="paragraph">
<p>In addition to the MirrorMaker 2 connectors, Kafka provides two connectors as examples:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>FileStreamSourceConnector</code> streams data from a file on the worker&#8217;s filesystem to Kafka, reading the input file and sending each line to a given Kafka topic.</p>
</li>
<li>
<p><code>FileStreamSinkConnector</code> streams data from Kafka to the worker&#8217;s filesystem, reading messages from a Kafka topic and writing a line for each in an output file.</p>
</li>
</ul>
</div>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>The following source connector diagram shows the process flow for a source connector that streams records from an external data system.
A Kafka Connect cluster might operate source and sink connectors at the same time.
Workers are running in distributed mode in the cluster.
Workers can run one or more tasks for more than one connector instance.</p>
</div>
<div class="paragraph">
<div class="title">Source connector streaming data to Kafka</div>
<p><span class="image"><img src="images/overview/kafka-concepts-source-connector.png" alt="Kafka Connect source connector worker interaction in distributed mode"></span></p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>A plugin provides the implementation artifacts for the source connector</p>
</li>
<li>
<p>A single worker initiates the source connector instance</p>
</li>
<li>
<p>The source connector creates the tasks to stream data</p>
</li>
<li>
<p>Tasks run in parallel to poll the external data system and return records</p>
</li>
<li>
<p>Transforms adjust the records, such as filtering or relabelling them</p>
</li>
<li>
<p>Converters put the records into a format suitable for Kafka</p>
</li>
<li>
<p>The source connector is managed using KafkaConnectors or the Kafka Connect API</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>The following sink connector diagram shows the process flow when streaming data from Kafka to an external data system.</p>
</div>
<div class="paragraph">
<div class="title">Sink connector streaming data from Kafka</div>
<p><span class="image"><img src="images/overview/kafka-concepts-sink-connector.png" alt="Kafka Connect sink connector worker interaction in distributed mode"></span></p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>A plugin provides the implementation artifacts for the sink connector</p>
</li>
<li>
<p>A single worker initiates the sink connector instance</p>
</li>
<li>
<p>The sink connector creates the tasks to stream data</p>
</li>
<li>
<p>Tasks run in parallel to poll Kafka and return records</p>
</li>
<li>
<p>Converters put the records into a format suitable for the external data system</p>
</li>
<li>
<p>Transforms adjust the records, such as filtering or relabelling them</p>
</li>
<li>
<p>The sink connector is managed using KafkaConnectors or the Kafka Connect API</p>
</li>
</ol>
</div>
</div>
<div class="sect3">
<h4 id="tasks"><a class="link" href="#tasks">4.1.2. Tasks</a></h4>
<div class="paragraph">
<p>Data transfer orchestrated by the Kafka Connect runtime is split into tasks that run in parallel.
A task is started using the configuration supplied by a connector instance.
Kafka Connect distributes the task configurations to workers, which instantiate and execute tasks.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>A source connector task polls the external data system and returns a list of records that a worker sends to the Kafka brokers.</p>
</li>
<li>
<p>A sink connector task receives Kafka records from a worker for writing to the external data system.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For sink connectors, the number of tasks created relates to the number of partitions being consumed.
For source connectors, how the source data is partitioned is defined by the connector.
You can control the maximum number of tasks that can run in parallel by setting <code>tasksMax</code> in the connector configuration.
The connector might create fewer tasks than the maximum setting.
For example, the connector might create fewer tasks if it&#8217;s not possible to split the source data into that many partitions.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<div class="title">Note</div>
</td>
<td class="content">
In the context of Kafka Connect, a <em>partition</em> can mean a topic partition or a <em>shard of data</em> in an external system.
</td>
</tr>
</table>
</div>
</div>
<div class="sect3">
<h4 id="workers"><a class="link" href="#workers">4.1.3. Workers</a></h4>
<div class="paragraph">
<p>Workers employ the connector configuration deployed to the Kafka Connect cluster.
The configuration is stored in an internal Kafka topic used by Kafka Connect.
Workers also run connectors and their tasks.</p>
</div>
<div class="paragraph">
<p>A Kafka Connect cluster contains a group of workers with the same <code>group.id</code>.
The ID identifies the cluster within Kafka.
The ID is assigned in the worker configuration through the <code>KafkaConnect</code> resource.
Worker configuration also specifies the names of internal Kafka Connect topics.
The topics store connector configuration, offset, and status information.
The group ID and names of these topics must also be unique to the Kafka Connect cluster.</p>
</div>
<div class="paragraph">
<p>Workers are assigned one or more connector instances and tasks.
The distributed approach to deploying Kafka Connect is fault tolerant and scalable.
If a worker pod fails, the tasks it was running are reassigned to active workers.
You can add to a group of worker pods through configuration of the <code>replicas</code> property in the <code>KafkaConnect</code> resource.</p>
</div>
</div>
<div class="sect3">
<h4 id="transforms"><a class="link" href="#transforms">4.1.4. Transforms</a></h4>
<div class="paragraph">
<p>Kafka Connect translates and transforms external data.
Single-message transforms change messages into a format suitable for the target destination.
For example, a transform might insert or rename a field. Transforms can also filter and route data.
Plugins contain the implementation required for workers to perform one or more transformations.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Source connectors apply transforms before converting data into a format supported by Kafka.</p>
</li>
<li>
<p>Sink connectors apply transforms after converting data into a format suitable for an external data system.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>A transform comprises a set of Java class files packaged in a JAR file for inclusion in a connector plugin.
Kafka Connect provides a set of standard transforms, but you can also create your own.</p>
</div>
</div>
<div class="sect3">
<h4 id="converters"><a class="link" href="#converters">4.1.5. Converters</a></h4>
<div class="paragraph">
<p>When a worker receives data, it converts the data into an appropriate format using a converter.
You specify converters for workers in the worker <code>config</code> in the <code>KafkaConnect</code> resource.</p>
</div>
<div class="paragraph">
<p>Kafka Connect can convert data to and from formats supported by Kafka, such as JSON or Avro.
It also supports schemas for structuring data.
If you are not converting data into a structured format, you don’t need to enable schemas.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<div class="title">Note</div>
</td>
<td class="content">
You can also specify converters for specific connectors to override the general Kafka Connect worker configuration that applies to all workers.
</td>
</tr>
</table>
</div>
<div class="ulist _additional-resources">
<div class="title">Additional resources</div>
<ul>
<li>
<p><a href="http://kafka.apache.org" target="_blank" rel="noopener">Apache Kafka documentation</a></p>
</li>
<li>
<p><a href="./configuring.html#property-kafka-connect-config-reference" target="_blank" rel="noopener">Kafka Connect configuration of workers</a></p>
</li>
<li>
<p><a href="./deploying.html#proc-mirrormaker-replication-str" target="_blank" rel="noopener">Synchronizing data between Kafka clusters using MirrorMaker 2</a></p>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="overview-components-kafka-bridge_str"><a class="link" href="#overview-components-kafka-bridge_str">5. Kafka Bridge interface</a></h2>
<div class="sectionbody">
<div class="paragraph _abstract">
<p>The Kafka Bridge provides a RESTful interface that allows HTTP-based clients to interact with a Kafka cluster. 
It offers the advantages of a web API connection to Strimzi, without the need for client applications to interpret the Kafka protocol.</p>
</div>
<div class="paragraph">
<p>The API has two main resources — <code>consumers</code> and <code>topics</code> — that are exposed and made accessible through endpoints to interact with consumers and producers in your Kafka cluster. The resources relate only to the Kafka Bridge, not the consumers and producers connected directly to Kafka.</p>
</div>
<div class="sect2">
<h3 id="http_requests"><a class="link" href="#http_requests">5.1. HTTP requests</a></h3>
<div class="paragraph">
<p>The Kafka Bridge supports HTTP requests to a Kafka cluster, with methods to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Send messages to a topic.</p>
</li>
<li>
<p>Retrieve messages from topics.</p>
</li>
<li>
<p>Retrieve a list of partitions for a topic.</p>
</li>
<li>
<p>Create and delete consumers.</p>
</li>
<li>
<p>Subscribe consumers to topics, so that they start receiving messages from those topics.</p>
</li>
<li>
<p>Retrieve a list of topics that a consumer is subscribed to.</p>
</li>
<li>
<p>Unsubscribe consumers from topics.</p>
</li>
<li>
<p>Assign partitions to consumers.</p>
</li>
<li>
<p>Commit a list of consumer offsets.</p>
</li>
<li>
<p>Seek on a partition, so that a consumer starts receiving messages from the first or last offset position, or a given offset position.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The methods provide JSON responses and HTTP response code error handling.
Messages can be sent in JSON or binary formats.</p>
</div>
<div class="paragraph">
<p>Clients can produce and consume messages without the requirement to use the native Kafka protocol.</p>
</div>
<div class="ulist _additional-resources">
<div class="title">Additional resources</div>
<ul>
<li>
<p>To view the API documentation, including example requests and responses, see <a href="https://strimzi.io/docs/bridge/latest/" target="_blank" rel="noopener">Using the Strimzi Kafka Bridge</a>.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="con-overview-components-kafka-bridge-clients_str"><a class="link" href="#con-overview-components-kafka-bridge-clients_str">5.2. Supported clients for the Kafka Bridge</a></h3>
<div class="paragraph">
<p>You can use the Kafka Bridge to integrate both <em>internal</em> and <em>external</em> HTTP client applications with your Kafka cluster.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Internal clients</dt>
<dd>
<p>Internal clients are container-based HTTP clients running in <em>the same</em> Kubernetes cluster as the Kafka Bridge itself.
Internal clients can access the Kafka Bridge on the host and port defined in the <code>KafkaBridge</code> custom resource.</p>
</dd>
<dt class="hdlist1">External clients</dt>
<dd>
<p>External clients are HTTP clients running <em>outside</em> the Kubernetes cluster in which the Kafka Bridge is deployed and running.
External clients can access the Kafka Bridge through an OpenShift Route, a loadbalancer service, or using an Ingress.</p>
</dd>
</dl>
</div>
<div class="paragraph">
<div class="title">HTTP internal and external client integration</div>
<p><span class="image"><img src="images/kafka-bridge.png" alt="Internal and external HTTP producers and consumers exchange data with the Kafka brokers through the Kafka Bridge"></span></p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="overview-components_str"><a class="link" href="#overview-components_str">6. Strimzi Operators</a></h2>
<div class="sectionbody">
<div class="paragraph _abstract">
<p>Operators are a method of packaging, deploying, and managing Kubernetes applications.
They provide a way to extend the Kubernetes API and simplify the administration tasks associated with specific applications.</p>
</div>
<div class="paragraph">
<p>Strimzi operators support tasks related to a Kafka deployment.
Strimzi custom resources provide the deployment configuration.
This includes configuration for Kafka clusters, topics, users, and other components.
Leveraging custom resource configuration, Strimzi operators create, configure, and manage Kafka components within a Kubernetes environment.
Using operators reduces the need for manual intervention and streamlines the process of managing Kafka in a Kubernetes cluster.</p>
</div>
<div class="paragraph">
<p>Strimzi provides the following operators for managing a Kafka cluster running within a Kubernetes cluster.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Cluster Operator</dt>
<dd>
<p>Deploys and manages Apache Kafka clusters, Kafka Connect, Kafka MirrorMaker, Kafka Bridge, Kafka Exporter, Cruise Control, and the Entity Operator</p>
</dd>
<dt class="hdlist1">Entity Operator</dt>
<dd>
<p>Comprises the Topic Operator and User Operator</p>
</dd>
<dt class="hdlist1">Topic Operator</dt>
<dd>
<p>Manages Kafka topics</p>
</dd>
<dt class="hdlist1">User Operator</dt>
<dd>
<p>Manages Kafka users</p>
</dd>
</dl>
</div>
<div class="paragraph">
<p>The Cluster Operator can deploy the Topic Operator and User Operator as part of an <strong>Entity Operator</strong> configuration at the same time as a Kafka cluster.</p>
</div>
<div class="paragraph">
<div class="title">Operators within the Strimzi architecture</div>
<p><span class="image"><img src="images/operators.png" alt="Operators within the Strimzi architecture"></span></p>
</div>
<div class="sect2">
<h3 id="overview-components-cluster-operator-str"><a class="link" href="#overview-components-cluster-operator-str">6.1. Cluster Operator</a></h3>
<div class="paragraph _abstract">
<p>Strimzi uses the Cluster Operator to deploy and manage clusters.
By default, when you deploy Strimzi a single Cluster Operator replica is deployed.
You can add replicas with leader election so that additional Cluster Operators are on standby in case of disruption.</p>
</div>
<div class="paragraph">
<p>The Cluster Operator manages the clusters of the following Kafka components:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Kafka (including ZooKeeper, Entity Operator, Kafka Exporter, and Cruise Control)</p>
</li>
<li>
<p>Kafka Connect</p>
</li>
<li>
<p>Kafka MirrorMaker</p>
</li>
<li>
<p>Kafka Bridge</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The clusters are deployed using custom resources.</p>
</div>
<div class="paragraph">
<p>For example, to deploy a Kafka cluster:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>A <code>Kafka</code> resource with the cluster configuration is created within the Kubernetes cluster.</p>
</li>
<li>
<p>The Cluster Operator deploys a corresponding Kafka cluster, based on what is declared in the <code>Kafka</code> resource.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The Cluster Operator can also deploy the following Strimzi operators through configuration of the <code>Kafka</code> resource:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Topic Operator to provide operator-style topic management through <code>KafkaTopic</code> custom resources</p>
</li>
<li>
<p>User Operator to provide operator-style user management through <code>KafkaUser</code> custom resources</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The Topic Operator and User Operator function within the Entity Operator on deployment.</p>
</div>
<div class="paragraph">
<p>You can use the Cluster Operator with a deployment of Strimzi Drain Cleaner to help with pod evictions.
By deploying the Strimzi Drain Cleaner, you can use the Cluster Operator to move Kafka pods instead of Kubernetes.
Strimzi Drain Cleaner annotates pods being evicted with a rolling update annotation.
The annotation informs the Cluster Operator to perform the rolling update.</p>
</div>
<div class="paragraph">
<div class="title">Example architecture for the Cluster Operator</div>
<p><span class="image"><img src="images/cluster-operator.png" alt="The Cluster Operator creates and deploys Kafka and ZooKeeper clusters"></span></p>
</div>
</div>
<div class="sect2">
<h3 id="overview-concepts-topic-operator-str"><a class="link" href="#overview-concepts-topic-operator-str">6.2. Topic Operator</a></h3>
<div class="paragraph _abstract">
<p>The Topic Operator provides a way of managing topics in a Kafka cluster through <code>KafkaTopic</code> resources.</p>
</div>
<div class="paragraph">
<div class="title">Example architecture for the Topic Operator</div>
<p><span class="image"><img src="images/topic-operator.png" alt="The Topic Operator manages topics for a Kafka cluster via KafkaTopic resources"></span></p>
</div>
<div class="paragraph">
<p>The Topic Operator manages Kafka topics by watching for <code>KafkaTopic</code> resources that describe Kafka topics, and ensuring that they are configured properly in the Kafka cluster.</p>
</div>
<div class="paragraph">
<p>When a <code>KafkaTopic</code> is created, deleted, or changed, the Topic Operator performs the corresponding action on the Kafka topic.</p>
</div>
<div class="paragraph">
<p>You can declare a <code>KafkaTopic</code> as part of your application&#8217;s deployment and the Topic Operator manages the Kafka topic for you.</p>
</div>
<div class="paragraph">
<p>The Topic Operator operates in the following modes:</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Unidirectional mode</dt>
<dd>
<p>Unidirectional mode means that the Topic Operator solely manages topics through the <code>KafkaTopic</code> resource. This mode does not require ZooKeeper and is compatible with using Strimzi in KRaft mode.</p>
</dd>
<dt class="hdlist1">Bidirectional mode</dt>
<dd>
<p>Bidirectional mode means that the Topic Operator can reconcile changes to a <code>KafkaTopic</code> resource to and from a Kafka cluster.
This means that you can update topics either through the <code>KafkaTopic</code> resource or directly in Kafka, and the Topic Operator will ensure that both sources are updated to reflect the changes. This mode requires ZooKeeper for cluster management.</p>
<div class="paragraph">
<p>The Topic Operator maintains information about each topic in a <em>topic store</em>, which is continually synchronized with updates from Kubernetes <code>KafkaTopic</code> custom resources or Kafka topics.
Updates from operations applied to a local in-memory topic store are persisted to a backup topic store on disk.</p>
</div>
</dd>
</dl>
</div>
</div>
<div class="sect2">
<h3 id="overview-concepts-user-operator-str"><a class="link" href="#overview-concepts-user-operator-str">6.3. User Operator</a></h3>
<div class="paragraph _abstract">
<p>The User Operator provides a way of managing users in a Kafka cluster through <code>KafkaUser</code> resources.</p>
</div>
<div class="paragraph">
<p>The User Operator manages Kafka users for a Kafka cluster by watching for <code>KafkaUser</code> resources that describe Kafka users,
and ensuring that they are configured properly in the Kafka cluster.</p>
</div>
<div class="paragraph">
<p>When a <code>KafkaUser</code> is created, deleted, or changed, the User Operator performs the corresponding action on the Kafka user.</p>
</div>
<div class="paragraph">
<p>You can declare a <code>KafkaUser</code> resource as part of your application&#8217;s deployment and the User Operator manages the Kafka user for you.
You can specify the authentication and authorization mechanism for the user.
You can also configure <em>user quotas</em> that control usage of Kafka resources to ensure, for example, that a user does not monopolize access to a broker.</p>
</div>
<div class="paragraph">
<p>When the user is created, the user credentials are created in a <code>Secret</code>.
Your application needs to use the user and its credentials for authentication and to produce or consume messages.</p>
</div>
<div class="paragraph">
<p>In addition to managing credentials for authentication, the User Operator also manages authorization rules by including a description of the user&#8217;s access rights in the <code>KafkaUser</code> declaration.</p>
</div>
</div>
<div class="sect2">
<h3 id="con-feature-gates-overview-str"><a class="link" href="#con-feature-gates-overview-str">6.4. Feature gates in Strimzi Operators</a></h3>
<div class="paragraph">
<p>You can enable and disable some features of operators using <em>feature gates</em>.</p>
</div>
<div class="paragraph">
<p>Feature gates are set in the operator configuration and have three stages of maturity: alpha, beta, or General Availability (GA).</p>
</div>
<div class="paragraph">
<p>For more information, see <a href="./deploying.html#ref-operator-cluster-feature-gates-str" target="_blank" rel="noopener">Feature gates</a>.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="configuration-points_str"><a class="link" href="#configuration-points_str">7. Kafka configuration</a></h2>
<div class="sectionbody">
<div class="paragraph _abstract">
<p>The deployment of Kafka components onto a Kubernetes cluster using Strimzi is highly configurable through the use of custom resources.
These resources are created as instances of APIs introduced by Custom Resource Definitions (CRDs), which extend Kubernetes resources.</p>
</div>
<div class="paragraph">
<p>CRDs act as configuration instructions to describe the custom resources in a Kubernetes cluster,
and are provided with Strimzi for each Kafka component used in a deployment, as well as users and topics.
CRDs and custom resources are defined as YAML files.
Example YAML files are provided with the Strimzi distribution.</p>
</div>
<div class="paragraph">
<p>CRDs also allow Strimzi resources to benefit from native Kubernetes features like CLI accessibility and configuration validation.</p>
</div>
<div class="paragraph">
<p>In this section we look at how Kafka components are configured through custom resources, starting with common configuration points and then important configuration considerations specific to components.</p>
</div>
<div class="paragraph">
<p>Strimzi provides <a href="./deploying.html#config-examples-str" target="_blank" rel="noopener">example configuration files</a>, which can serve as a starting point when building your own Kafka component configuration for deployment.</p>
</div>
<div class="sect2">
<h3 id="configuration-points-resources_str"><a class="link" href="#configuration-points-resources_str">7.1. Custom resources</a></h3>
<div class="paragraph">
<p>After a new custom resource type is added to your cluster by installing a CRD, you can create instances of the resource based on its specification.</p>
</div>
<div class="paragraph">
<p>The custom resources for Strimzi components have common configuration properties, which are defined under <code>spec</code>.</p>
</div>
<div class="paragraph">
<p>In this fragment from a Kafka topic custom resource, the <code>apiVersion</code> and <code>kind</code> properties identify the associated CRD.
The <code>spec</code> property shows configuration that defines the number of partitions and replicas for the topic.</p>
</div>
<h4 id="kafka_topic_custom_resource" class="discrete">Kafka topic custom resource</h4>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
  name: my-topic
  labels:
    strimzi.io/cluster: my-cluster
spec:
  partitions: 1
  replicas: 1
  # ...</code></pre>
</div>
</div>
<div class="paragraph">
<p>There are many additional configuration options that can be incorporated into a YAML definition, some common and some specific to a particular component.</p>
</div>
<div class="ulist _additional-resources">
<div class="title">Additional resources</div>
<ul>
<li>
<p><a href="https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/" target="_blank" rel="noopener">Extend the Kubernetes API with CustomResourceDefinitions</a></p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="configuration-points-common_str"><a class="link" href="#configuration-points-common_str">7.2. Common configuration</a></h3>
<div class="paragraph">
<p>Some of the configuration options common to resources are described here.
<a href="#security-overview_str">Security</a> and <a href="#metrics-overview_str">metrics collection</a> might also be adopted where applicable.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Bootstrap servers</dt>
<dd>
<p>Bootstrap servers are used for host/port connection to a Kafka cluster for:</p>
<div class="ulist">
<ul>
<li>
<p>Kafka Connect</p>
</li>
<li>
<p>Kafka Bridge</p>
</li>
<li>
<p>Kafka MirrorMaker producers and consumers</p>
</li>
</ul>
</div>
</dd>
<dt class="hdlist1">CPU and memory resources</dt>
<dd>
<p>You request CPU and memory resources for components. Limits specify the maximum resources that can be consumed by a given container.</p>
<div class="paragraph">
<p>Resource requests and limits for the Topic Operator and User Operator are set in the <code>Kafka</code> resource.</p>
</div>
</dd>
<dt class="hdlist1">Logging</dt>
<dd>
<p>You define the logging level for the component. Logging can be defined directly (inline) or externally using a config map.</p>
</dd>
<dt class="hdlist1">Healthchecks</dt>
<dd>
<p>Healthcheck configuration introduces <em>liveness</em> and <em>readiness</em> probes to know when to restart a container (liveness) and when a container can accept traffic (readiness).</p>
</dd>
<dt class="hdlist1">JVM options</dt>
<dd>
<p>JVM options provide maximum and minimum memory allocation to optimize the performance of the component according to the platform it is running on.</p>
</dd>
<dt class="hdlist1">Pod scheduling</dt>
<dd>
<p>Pod schedules use <em>affinity/anti-affinity</em> rules to determine under what circumstances a pod is scheduled onto a node.</p>
</dd>
</dl>
</div>
<h4 id="example_yaml_showing_common_configuration" class="discrete">Example YAML showing common configuration</h4>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-cluster
spec:
  # ...
  bootstrapServers: my-cluster-kafka-bootstrap:9092
  resources:
    requests:
      cpu: 12
      memory: 64Gi
    limits:
      cpu: 12
      memory: 64Gi
  logging:
    type: inline
    loggers:
      connect.root.logger.level: INFO
  readinessProbe:
    initialDelaySeconds: 15
    timeoutSeconds: 5
  livenessProbe:
    initialDelaySeconds: 15
    timeoutSeconds: 5
  jvmOptions:
    "-Xmx": "2g"
    "-Xms": "2g"
  template:
    pod:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: node-type
                    operator: In
                    values:
                      - fast-network
  # ...</code></pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="configuration-points-broker_str"><a class="link" href="#configuration-points-broker_str">7.3. Kafka cluster configuration</a></h3>
<div class="paragraph">
<p>A kafka cluster comprises one or more brokers.
For producers and consumers to be able to access topics within the brokers, Kafka configuration must define how data is stored in the cluster, and how the data is accessed.
You can configure a Kafka cluster to run with multiple broker nodes across racks.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Storage</dt>
<dd>
<p>Kafka and ZooKeeper store data on disks.</p>
<div class="paragraph">
<p>Strimzi requires block storage provisioned through <code>StorageClass</code>.
The file system format for storage must be <em>XFS</em> or <em>EXT4</em>.
Three types of data storage are supported:</p>
</div>
<div class="openblock">
<div class="content">
<div class="dlist">
<dl>
<dt class="hdlist1">Ephemeral (Recommended for development only)</dt>
<dd>
<p>Ephemeral storage stores data for the lifetime of an instance. Data is lost when the instance is restarted.</p>
</dd>
<dt class="hdlist1">Persistent</dt>
<dd>
<p>Persistent storage relates to long-term data storage independent of the lifecycle of the instance.</p>
</dd>
<dt class="hdlist1">JBOD (Just a Bunch of Disks, suitable for Kafka only)</dt>
<dd>
<p>JBOD allows you to use multiple disks to store commit logs in each broker.</p>
</dd>
</dl>
</div>
</div>
</div>
<div class="paragraph">
<p>The disk capacity used by an existing Kafka cluster can be increased if supported by the infrastructure.</p>
</div>
</dd>
<dt class="hdlist1">Listeners</dt>
<dd>
<p>Listeners configure how clients connect to a Kafka cluster.</p>
<div class="paragraph">
<p>By specifying a unique name and port for each listener within a Kafka cluster,
you can configure multiple listeners.</p>
</div>
<div class="paragraph">
<p>The following types of listener are supported:</p>
</div>
<div class="openblock">
<div class="content">
<div class="ulist">
<ul>
<li>
<p><strong>Internal listeners</strong> for access within Kubernetes</p>
</li>
<li>
<p><strong>External listeners</strong> for access outside of Kubernetes</p>
</li>
</ul>
</div>
</div>
</div>
<div class="paragraph">
<p>You can enable TLS encryption for listeners, and configure <a href="#security-configuration-authentication_str">authentication</a>.</p>
</div>
<div class="paragraph">
<p>Internal listeners expose Kafka by specifying an <code>internal</code> type:</p>
</div>
<div class="openblock">
<div class="content">
<div class="ulist">
<ul>
<li>
<p><code>internal</code> to connect within the same Kubernetes cluster</p>
</li>
<li>
<p><code>cluster-ip</code> to expose Kafka using per-broker <code>ClusterIP</code> services</p>
</li>
</ul>
</div>
</div>
</div>
<div class="paragraph">
<p>External listeners expose Kafka by specifying an external <code>type</code>:</p>
</div>
<div class="openblock">
<div class="content">
<div class="ulist">
<ul>
<li>
<p><code>route</code> to use OpenShift routes and the default HAProxy router</p>
</li>
<li>
<p><code>loadbalancer</code> to use loadbalancer services</p>
</li>
<li>
<p><code>nodeport</code> to use ports on Kubernetes nodes</p>
</li>
<li>
<p><code>ingress</code> to use Kubernetes <em>Ingress</em> and the <a href="https://github.com/kubernetes/ingress-nginx" target="_blank" rel="noopener">Ingress NGINX Controller for Kubernetes</a></p>
</li>
</ul>
</div>
</div>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<div class="title">Note</div>
</td>
<td class="content">
With the <code>cluster-ip</code> type can add your own access mechanism.
For example, you can use the listener with a custom Ingress controller or the Kubernetes Gateway API.
</td>
</tr>
</table>
</div>
</dd>
</dl>
</div>
<div class="paragraph">
<p>If you are using <a href="#security-configuration-authentication_str">OAuth 2.0 for token-based authentication</a>, you can configure listeners to use the authorization server.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Rack awareness</dt>
<dd>
<p>Racks represent data centers, or racks in data centers, or availability zones.
Configure rack awareness to distribute Kafka broker pods and topic replicas across racks.
Enable rack awareness using the <code>rack</code> property to specify a <code>topologyKey</code>.
The <code>topologyKey</code> is the name of the label assigned to Kubernetes worker nodes, which identifies the rack.
Strimzi assigns a rack ID to each Kafka broker.
Kafka brokers use the IDs to spread partition replicas across racks.
You can also specify the <code>RackAwareReplicaSelector</code> selector plugin to use with rack awareness.
The plugin matches the rack IDs of brokers and consumers, so that messages are consumed from the closest replica.
To use the plugin, consumers must also have rack awareness enabled.
You can enable rack awareness in Kafka Connect, MirrorMaker 2, and the Kafka Bridge.</p>
</dd>
</dl>
</div>
<h4 id="example_yaml_showing_kafka_configuration" class="discrete">Example YAML showing Kafka configuration</h4>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-shell hljs" data-lang="shell">apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    # ...
    listeners:
      - name: tls
        port: 9093
        type: internal
        tls: true
        authentication:
          type: tls
      - name: external1
        port: 9094
        type: route
        tls: true
        authentication:
          type: tls
    # ...
    storage:
      type: persistent-claim
      size: 10000Gi
    # ...
    rack:
      topologyKey: topology.kubernetes.io/zone
    config:
      replica.selector.class: org.apache.kafka.common.replica.RackAwareReplicaSelector
    # ...</code></pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="configuration-points-node_pools_str"><a class="link" href="#configuration-points-node_pools_str">7.4. Kafka node pools configuration</a></h3>
<div class="paragraph _abstract">
<p>A node pool refers to a distinct group of Kafka nodes within a Kafka cluster.
By using node pools, nodes can have different configuration within the same Kafka cluster.
Configuration options not specified in the node pool are inherited from the Kafka configuration.</p>
</div>
<div class="paragraph">
<p>You can deploy a Kafka cluster with one or more node pools.
The node pool configuration includes mandatory and optional settings.
Configuration for replicas, roles, and storage is mandatory.</p>
</div>
<div class="paragraph">
<p>If you are using KRaft mode, you can specify roles that all nodes in the node pool operate as brokers, controllers, or both.
Controller and dual roles are specific to KRaft.
If you are using Kafka clusters that use ZooKeeper for cluster management, you can use node pools that are configured with broker roles only.</p>
</div>
<h4 id="example_yaml_showing_node_pool_configuration" class="discrete">Example YAML showing node pool configuration</h4>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-shell hljs" data-lang="shell">apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: pool-a
  labels:
    strimzi.io/cluster: my-cluster
spec:
  replicas: 3
  roles:
    - broker
  storage:
    type: jbod
    volumes:
      - id: 0
        type: persistent-claim
        size: 100Gi
        deleteClaim: false</code></pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="con-configuration-points-mm2-str"><a class="link" href="#con-configuration-points-mm2-str">7.5. Kafka MirrorMaker 2 configuration</a></h3>
<div class="paragraph _abstract">
<p>Kafka MirrorMaker 2 replicates data between two or more active Kafka clusters, within or across data centers.
To set up MirrorMaker 2, a source and target (destination) Kafka cluster must be running.</p>
</div>
<div class="paragraph">
<p>The process of <em>mirroring</em> data from a source cluster to a target cluster is asynchronous.
Each MirrorMaker 2 instance mirrors data from one source cluster to one target cluster.
You can use more than one MirrorMaker 2 instance to mirror data between any number of clusters.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="images/mirrormaker.png" alt="MirrorMaker 2 replication">
</div>
<div class="title">Figure 1. Replication across two clusters</div>
</div>
<div class="paragraph">
<p>MirrorMaker 2 uses source and target cluster configuration as follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Source cluster configuration for consuming data from the source cluster</p>
</li>
<li>
<p>Target cluster configuration for outputting data to the target cluster</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Topic and consumer group replication is specified as comma-separated lists or regular expression patterns.</p>
</div>
<h4 id="example_yaml_showing_mirrormaker_2_configuration" class="discrete">Example YAML showing MirrorMaker 2 configuration</h4>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">  apiVersion: kafka.strimzi.io/v1beta2
  kind: KafkaMirrorMaker2
  metadata:
    name: my-mirror-maker2
    spec:
      version: 3.7.0
      connectCluster: "my-cluster-target"
      clusters:
      - alias: "my-cluster-source"
        bootstrapServers: my-cluster-source-kafka-bootstrap:9092
      - alias: "my-cluster-target"
        bootstrapServers: my-cluster-target-kafka-bootstrap:9092
      mirrors:
      - sourceCluster: "my-cluster-source"
        targetCluster: "my-cluster-target"
        sourceConnector: {}
      topicsPattern: ".*"
      groupsPattern: "group1|group2|group3"</code></pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="con-configuration-points-mm-str"><a class="link" href="#con-configuration-points-mm-str">7.6. Kafka MirrorMaker configuration</a></h3>
<div class="paragraph">
<p>Kafka MirrorMaker (also referred to as MirrorMaker 1) uses producers and consumers to replicate data across clusters as follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Consumers consume data from the source cluster</p>
</li>
<li>
<p>Producers output data to the target cluster</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Consumer and producer configuration includes any required authentication and encryption settings.
An <code>include</code> property defines the topics to mirror from the source to the target cluster.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<div class="title">Note</div>
</td>
<td class="content">
MirrorMaker was deprecated in Kafka 3.0.0 and will be removed in Kafka 4.0.0.
As a consequence, the Strimzi <code>KafkaMirrorMaker</code> custom resource which is used to deploy MirrorMaker has been deprecated.
The <code>KafkaMirrorMaker</code> resource will be removed from Strimzi when Kafka 4.0.0 is adopted.
</td>
</tr>
</table>
</div>
<h4 id="key_consumer_configuration" class="discrete">Key Consumer configuration</h4>
<div class="dlist">
<dl>
<dt class="hdlist1">Consumer group identifier</dt>
<dd>
<p>The consumer group ID for a MirrorMaker consumer so that messages consumed are assigned to a consumer group.</p>
</dd>
<dt class="hdlist1">Number of consumer streams</dt>
<dd>
<p>A value to determine the number of consumers in a consumer group that consume a message in parallel.</p>
</dd>
<dt class="hdlist1">Offset commit interval</dt>
<dd>
<p>An offset commit interval to set the time between consuming and committing a message.</p>
</dd>
</dl>
</div>
<h4 id="key_producer_configuration" class="discrete">Key Producer configuration</h4>
<div class="dlist">
<dl>
<dt class="hdlist1">Cancel option for send failure</dt>
<dd>
<p>You can define whether a message send failure is ignored or MirrorMaker is terminated and recreated.</p>
</dd>
</dl>
</div>
<h4 id="example_yaml_showing_mirrormaker_configuration" class="discrete">Example YAML showing MirrorMaker configuration</h4>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaMirrorMaker
metadata:
  name: my-mirror-maker
spec:
  # ...
  consumer:
    bootstrapServers: my-source-cluster-kafka-bootstrap:9092
    groupId: "my-group"
    numStreams: 2
    offsetCommitInterval: 120000
    # ...
  producer:
    # ...
    abortOnSendFailure: false
    # ...
  include: "my-topic|other-topic"
  # ...</code></pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="configuration-points-connect_str"><a class="link" href="#configuration-points-connect_str">7.7. Kafka Connect configuration</a></h3>
<div class="paragraph _abstract">
<p>Use Strimzi’s <code>KafkaConnect</code> resource to quickly and easily create new Kafka Connect clusters.</p>
</div>
<div class="paragraph">
<p>When you deploy Kafka Connect using the <code>KafkaConnect</code> resource,
you specify bootstrap server addresses (in <code>spec.bootstrapServers</code>) for connecting to a Kafka cluster.
You can specify more than one address in case a server goes down.
You also specify the authentication credentials and TLS encryption certificates to make a secure connection.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<div class="title">Note</div>
</td>
<td class="content">
The Kafka cluster doesn&#8217;t need to be managed by Strimzi or deployed to a Kubernetes cluster.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>You can also use the <code>KafkaConnect</code> resource to specify the following:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Plugin configuration to build a container image that includes the plugins to make connections</p>
</li>
<li>
<p>Configuration for the worker pods that belong to the Kafka Connect cluster</p>
</li>
<li>
<p>An annotation to enable use of the <code>KafkaConnector</code> resource to manage plugins</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The Cluster Operator manages Kafka Connect clusters deployed using the <code>KafkaConnect</code> resource and connectors created using the <code>KafkaConnector</code> resource.</p>
</div>
<h4 id="plugin_configuration" class="discrete">Plugin configuration</h4>
<div class="paragraph">
<p>Plugins provide the implementation for creating connector instances.
When a plugin is instantiated, configuration is provided for connection to a specific type of external data system.
Plugins provide a set of one or more JAR files that define a connector and task implementation for connecting to a given kind of data source.
Plugins for many external systems are available for use with Kafka Connect.
You can also create your own plugins.</p>
</div>
<div class="paragraph">
<p>The configuration describes the source input data and target output data to feed into and out of Kafka Connect.
For a source connector, external source data must reference specific topics that will store the messages.
The plugins might also contain the libraries and files needed to transform the data.</p>
</div>
<div class="paragraph">
<p>A Kafka Connect deployment can have one or more plugins, but only one version of each plugin.</p>
</div>
<div class="paragraph">
<p>You can create a custom Kafka Connect image that includes your choice of plugins.
You can create the image in two ways:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="./deploying.html#creating-new-image-using-kafka-connect-build-str" target="_blank" rel="noopener">Automatically using Kafka Connect configuration</a></p>
</li>
<li>
<p><a href="./deploying.html#creating-new-image-from-base-str" target="_blank" rel="noopener">Manually using a Dockerfile and a Kafka container image as a base image</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>To create the container image automatically, you specify the plugins to add to your Kafka Connect cluster using the <code>build</code> property of the <code>KafkaConnect</code> resource.
Strimzi automatically downloads and adds the plugin artifacts to a new container image.</p>
</div>
<div class="listingblock">
<div class="title">Example plugin configuration</div>
<div class="content">
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-connect-cluster
  annotations:
    strimzi.io/use-connector-resources: "true"
spec:
  # ...
  build: # <b class="conum">(1)</b>
    output: # <b class="conum">(2)</b>
      type: docker
      image: my-registry.io/my-org/my-connect-cluster:latest
      pushSecret: my-registry-credentials
    plugins: # <b class="conum">(3)</b>
      - name: my-connector
        artifacts:
          - type: tgz
            url: https://&lt;plugin_download_location&gt;.tgz
            sha512sum: &lt;checksum_to_verify_the_plugin&gt;
      # ...
  # ...</code></pre>
</div>
</div>
<div class="colist arabic">
<ol>
<li>
<p><a href="./configuring.html#type-Build-reference" target="_blank" rel="noopener">Build configuration properties</a> for building a container image with plugins automatically.</p>
</li>
<li>
<p>Configuration of the container registry where new images are pushed. The <code>output</code> properties describe the type and name of the image, and optionally the name of the secret containing the credentials needed to access the container registry.</p>
</li>
<li>
<p>List of plugins and their artifacts to add to the new container image. The <code>plugins</code> properties describe the type of artifact and the URL from which the artifact is downloaded. Each plugin must be configured with at least one artifact. Additionally, you can specify a SHA-512 checksum to verify the artifact before unpacking it.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>If you are using a Dockerfile to build an image, you can use Strimzi’s latest container image as a base image to add your plugin configuration file.</p>
</div>
<div class="listingblock">
<div class="title">Example showing manual addition of plugin configuration</div>
<div class="content">
<pre class="highlightjs highlight"><code class="language-none hljs">FROM quay.io/strimzi/kafka:0.40.0-kafka-3.7.0
USER root:root
COPY ./<em>my-plugins</em>/ /opt/kafka/plugins/
USER 1001</code></pre>
</div>
</div>
<h4 id="kafka_connect_cluster_configuration_for_workers" class="discrete">Kafka Connect cluster configuration for workers</h4>
<div class="paragraph">
<p>You specify the configuration for workers in the <code>config</code> property of the <code>KafkaConnect</code> resource.</p>
</div>
<div class="paragraph">
<p>A distributed Kafka Connect cluster has a group ID and a set of internal configuration topics.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>group.id</code></p>
</li>
<li>
<p><code>offset.storage.topic</code></p>
</li>
<li>
<p><code>config.storage.topic</code></p>
</li>
<li>
<p><code>status.storage.topic</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Kafka Connect clusters are configured by default with the same values for these properties.
Kafka Connect clusters cannot share the group ID or topic names as it will create errors.
If multiple different Kafka Connect clusters are used, these settings must be unique for the workers of each Kafka Connect cluster created.</p>
</div>
<div class="paragraph">
<p>The names of the connectors used by each Kafka Connect cluster must also be unique.</p>
</div>
<div class="paragraph">
<p>In the following example worker configuration, JSON converters are specified.
A replication factor is set for the internal Kafka topics used by Kafka Connect.
This should be at least 3 for a production environment.
Changing the replication factor after the topics have been created will have no effect.</p>
</div>
<div class="listingblock">
<div class="title">Example worker configuration</div>
<div class="content">
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
# ...
spec:
  config:
    # ...
    group.id: my-connect-cluster <b class="conum">(1)</b>
    offset.storage.topic: my-connect-cluster-offsets <b class="conum">(2)</b>
    config.storage.topic: my-connect-cluster-configs <b class="conum">(3)</b>
    status.storage.topic: my-connect-cluster-status <b class="conum">(4)</b>
    key.converter: org.apache.kafka.connect.json.JsonConverter <b class="conum">(5)</b>
    value.converter: org.apache.kafka.connect.json.JsonConverter <b class="conum">(6)</b>
    key.converter.schemas.enable: true <b class="conum">(7)</b>
    value.converter.schemas.enable: true <b class="conum">(8)</b>
    config.storage.replication.factor: 3 <b class="conum">(9)</b>
    offset.storage.replication.factor: 3 <b class="conum">(10)</b>
    status.storage.replication.factor: 3 <b class="conum">(11)</b>
  # ...</code></pre>
</div>
</div>
<div class="colist arabic">
<ol>
<li>
<p>The Kafka Connect cluster ID within Kafka. Must be unique for each Kafka Connect cluster.</p>
</li>
<li>
<p>Kafka topic that stores connector offsets. Must be unique for each Kafka Connect cluster.</p>
</li>
<li>
<p>Kafka topic that stores connector and task status configurations. Must be unique for each Kafka Connect cluster.</p>
</li>
<li>
<p>Kafka topic that stores connector and task status updates. Must be unique for each Kafka Connect cluster.</p>
</li>
<li>
<p>Converter to transform message keys into JSON format for storage in Kafka.</p>
</li>
<li>
<p>Converter to transform message values into JSON format for storage in Kafka.</p>
</li>
<li>
<p>Schema enabled for converting message keys into structured JSON format.</p>
</li>
<li>
<p>Schema enabled for converting message values into structured JSON format.</p>
</li>
<li>
<p>Replication factor for the Kafka topic that stores connector offsets.</p>
</li>
<li>
<p>Replication factor for the Kafka topic that stores connector and task status configurations.</p>
</li>
<li>
<p>Replication factor for the Kafka topic that stores connector and task status updates.</p>
</li>
</ol>
</div>
<h4 id="kafkaconnector_management_of_connectors" class="discrete"><code>KafkaConnector</code> management of connectors</h4>
<div class="paragraph">
<p>After plugins have been added to the container image used for the worker pods in a deployment,
you can use Strimzi’s <code>KafkaConnector</code> custom resource or the Kafka Connect API to manage connector instances.
You can also create new connector instances using these options.</p>
</div>
<div class="paragraph">
<p>The <code>KafkaConnector</code> resource offers a Kubernetes-native approach to management of connectors by the Cluster Operator.
To manage connectors with <code>KafkaConnector</code> resources, you must specify an annotation in your <code>KafkaConnect</code> custom resource.</p>
</div>
<div class="listingblock">
<div class="title">Annotation to enable KafkaConnectors</div>
<div class="content">
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-connect-cluster
  annotations:
    strimzi.io/use-connector-resources: "true"
  # ...</code></pre>
</div>
</div>
<div class="paragraph">
<p>Setting <code>use-connector-resources</code> to <code>true</code>  enables KafkaConnectors to create, delete, and reconfigure connectors.</p>
</div>
<div class="paragraph">
<p>If <code>use-connector-resources</code> is enabled in your <code>KafkaConnect</code> configuration, you must use the <code>KafkaConnector</code> resource to define and manage connectors.
<code>KafkaConnector</code> resources are configured to connect to external systems.
They are deployed to the same Kubernetes cluster as the Kafka Connect cluster and Kafka cluster interacting with the external data system.</p>
</div>
<div class="paragraph">
<div class="title">Kafka components are contained in the same Kubernetes cluster</div>
<p><span class="image"><img src="images/overview/kafka-concepts-kafka-connector.png" alt="Kafka and Kafka Connect clusters"></span></p>
</div>
<div class="paragraph">
<p>The configuration specifies how connector instances connect to an external data system, including any authentication.
You also need to state what data to watch.
For a source connector, you might provide a database name in the configuration.
You can also specify where the data should sit in Kafka by specifying a target topic name.</p>
</div>
<div class="paragraph">
<p>Use <code>tasksMax</code> to specify the maximum number of tasks.
For example, a source connector with <code>tasksMax: 2</code> might split the import of source data into two tasks.</p>
</div>
<div class="listingblock">
<div class="title">Example KafkaConnector source connector configuration</div>
<div class="content">
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: my-source-connector  # <b class="conum">(1)</b>
  labels:
    strimzi.io/cluster: my-connect-cluster # <b class="conum">(2)</b>
spec:
  class: org.apache.kafka.connect.file.FileStreamSourceConnector # <b class="conum">(3)</b>
  tasksMax: 2 # <b class="conum">(4)</b>
  autoRestart: # <b class="conum">(5)</b>
    enabled: true
  config: # <b class="conum">(6)</b>
    file: "/opt/kafka/LICENSE" # <b class="conum">(7)</b>
    topic: my-topic # <b class="conum">(8)</b>
    # ...</code></pre>
</div>
</div>
<div class="colist arabic">
<ol>
<li>
<p>Name of the <code>KafkaConnector</code> resource, which is used as the name of the connector. Use any name that is valid for a Kubernetes resource.</p>
</li>
<li>
<p>Name of the Kafka Connect cluster to create the connector instance in. Connectors must be deployed to the same namespace as the Kafka Connect cluster they link to.</p>
</li>
<li>
<p>Full name of the connector class. This should be present in the image being used by the Kafka Connect cluster.</p>
</li>
<li>
<p>Maximum number of Kafka Connect tasks that the connector can create.</p>
</li>
<li>
<p>Enables automatic restarts of failed connectors and tasks. By default, the number of restarts is indefinite, but you can set a maximum on the number of automatic restarts using the <code>maxRestarts</code> property.</p>
</li>
<li>
<p><a href="./deploying.html#kafkaconnector-configs" target="_blank" rel="noopener">Connector configuration</a> as key-value pairs.</p>
</li>
<li>
<p>Location of the external data file. In this example, we&#8217;re configuring the <code>FileStreamSourceConnector</code> to read from the <code>/opt/kafka/LICENSE</code> file.</p>
</li>
<li>
<p>Kafka topic to publish the source data to.</p>
</li>
</ol>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<div class="title">Note</div>
</td>
<td class="content">
You can <a href="./deploying.html#assembly-loading-config-with-providers-str" target="_blank" rel="noopener">load confidential configuration values for a connector</a> from external sources, such as Kubernetes Secrets or ConfigMaps.
</td>
</tr>
</table>
</div>
<h4 id="kafka_connect_api" class="discrete">Kafka Connect API</h4>
<div class="paragraph">
<p>Use the Kafka Connect REST API as an alternative to using <code>KafkaConnector</code> resources to manage connectors.
The Kafka Connect REST API is available as a service running on <code><em>&lt;connect_cluster_name&gt;</em>-connect-api:8083</code>, where <em>&lt;connect_cluster_name&gt;</em> is the name of your Kafka Connect cluster.</p>
</div>
<div class="paragraph">
<p>You add the connector configuration as a JSON object.</p>
</div>
<div class="listingblock">
<div class="title">Example curl request to add connector configuration</div>
<div class="content">
<pre class="highlightjs highlight"><code class="language-curl hljs" data-lang="curl">curl -X POST \
  http://my-connect-cluster-connect-api:8083/connectors \
  -H 'Content-Type: application/json' \
  -d '{ "name": "my-source-connector",
    "config":
    {
      "connector.class":"org.apache.kafka.connect.file.FileStreamSourceConnector",
      "file": "/opt/kafka/LICENSE",
      "topic":"my-topic",
      "tasksMax": "4",
      "type": "source"
    }
}'</code></pre>
</div>
</div>
<div class="paragraph">
<p>If KafkaConnectors are enabled, manual changes made directly using the Kafka Connect REST API are reverted by the Cluster Operator.</p>
</div>
<div class="paragraph">
<p>The operations supported by the REST API are described in the <a href="https://kafka.apache.org/documentation#connect_rest" target="_blank" rel="noopener">Apache Kafka Connect API documentation</a>.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<div class="title">Note</div>
</td>
<td class="content">
You can expose the Kafka Connect API service outside Kubernetes.
You do this by creating a service that uses a connection mechanism that provides the access, such as an ingress or route.
Use advisedly as the connection is insecure.
</td>
</tr>
</table>
</div>
<div class="ulist _additional-resources">
<div class="title">Additional resources</div>
<ul>
<li>
<p><a href="./deploying.html#con-kafka-connect-config-str" target="_blank" rel="noopener">Kafka Connect configuration options</a></p>
</li>
<li>
<p><a href="./deploying.html#con-kafka-connect-multiple-instances-str" target="_blank" rel="noopener">Kafka Connect configuration for multiple instances</a></p>
</li>
<li>
<p><a href="./deploying.html#using-kafka-connect-with-plug-ins-str" target="_blank" rel="noopener">Extending Kafka Connect with plugins</a></p>
</li>
<li>
<p><a href="./deploying.html#creating-new-image-using-kafka-connect-build-str" target="_blank" rel="noopener">Creating a new container image automatically using Strimzi</a></p>
</li>
<li>
<p><a href="./deploying.html#creating-new-image-from-base-str" target="_blank" rel="noopener">Creating a Docker image from the Kafka Connect base image</a></p>
</li>
<li>
<p><a href="./configuring.html#type-Build-reference" target="_blank" rel="noopener">Build schema reference</a></p>
</li>
<li>
<p><a href="./deploying.html#kafkaconnector-configs" target="_blank" rel="noopener">Source and sink connector configuration options</a></p>
</li>
<li>
<p><a href="./deploying.html#assembly-loading-config-with-providers-str" target="_blank" rel="noopener">Loading configuration values from external sources</a></p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="configuration-points-bridge_str"><a class="link" href="#configuration-points-bridge_str">7.8. Kafka Bridge configuration</a></h3>
<div class="paragraph">
<p>A Kafka Bridge configuration requires a bootstrap server specification for the Kafka cluster it connects to, as well as any encryption and authentication options required.</p>
</div>
<div class="paragraph">
<p>Kafka Bridge consumer and producer configuration is standard, as described in the <a href="https://kafka.apache.org/documentation/#consumerconfigs" target="_blank" rel="noopener">Apache Kafka configuration documentation for consumers</a> and <a href="https://kafka.apache.org/documentation/#producerconfigs" target="_blank" rel="noopener">Apache Kafka configuration documentation for producers</a>.</p>
</div>
<div class="paragraph">
<p>HTTP-related configuration options set the port connection which the server listens on.</p>
</div>
<h4 id="cors" class="discrete">CORS</h4>
<div class="paragraph">
<p>The Kafka Bridge supports the use of Cross-Origin Resource Sharing (CORS).
CORS is a HTTP mechanism that allows browser access to selected resources from more than one origin, for example, resources on different domains.
If you choose to use CORS, you can define a list of allowed resource origins and HTTP methods for interaction with the Kafka cluster through the Kafka Bridge.
The lists are defined in the <code>http</code> specification of the Kafka Bridge configuration.</p>
</div>
<div class="paragraph">
<p>CORS allows for <em>simple</em> and <em>preflighted</em> requests between origin sources on different domains.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>A simple request is a HTTP request that must have an allowed origin defined in its header.</p>
</li>
<li>
<p>A preflighted request sends an initial OPTIONS HTTP request before the actual request to check that the origin and the method are allowed.</p>
</li>
</ul>
</div>
<h4 id="example_yaml_showing_kafka_bridge_configuration" class="discrete">Example YAML showing Kafka Bridge configuration</h4>
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaBridge
metadata:
  name: my-bridge
spec:
  # ...
  bootstrapServers: my-cluster-kafka:9092
  http:
    port: 8080
    cors:
      allowedOrigins: "https://strimzi.io"
      allowedMethods: "GET,POST,PUT,DELETE,OPTIONS,PATCH"
  consumer:
    config:
      auto.offset.reset: earliest
  producer:
    config:
      delivery.timeout.ms: 300000
  # ...</code></pre>
</div>
</div>
<div class="ulist _additional-resources">
<div class="title">Additional resources</div>
<ul>
<li>
<p><a href="https://fetch.spec.whatwg.org/" target="_blank" rel="noopener">Fetch CORS specification</a></p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="security-overview_str"><a class="link" href="#security-overview_str">8. Securing Kafka</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>A secure deployment of Strimzi might encompass one or more of the following security measures:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Encryption for data exchange</p>
</li>
<li>
<p>Authentication to prove identity</p>
</li>
<li>
<p>Authorization to allow or decline actions executed by users</p>
</li>
<li>
<p>Running Strimzi on FIPS-enabled Kubernetes clusters to ensure data security and system interoperability</p>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="security-configuration-encryption_str"><a class="link" href="#security-configuration-encryption_str">8.1. Encryption</a></h3>
<div class="paragraph">
<p>Strimzi supports Transport Layer Security (TLS), a protocol for encrypted communication.</p>
</div>
<div class="paragraph">
<p>Communication is always encrypted for communication between:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Kafka brokers</p>
</li>
<li>
<p>ZooKeeper nodes</p>
</li>
<li>
<p>Kafka brokers and ZooKeeper nodes</p>
</li>
<li>
<p>Operators and Kafka brokers</p>
</li>
<li>
<p>Operators and ZooKeeper nodes</p>
</li>
<li>
<p>Kafka Exporter</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>You can also configure TLS encryption between Kafka brokers and clients.
TLS is specified for external clients when configuring an external listener for the Kafka broker.</p>
</div>
<div class="paragraph">
<p>Strimzi components and Kafka clients use digital certificates for encryption.
The Cluster Operator sets up certificates to enable encryption within the Kafka cluster.
You can provide your own server certificates, referred to as <em>Kafka listener certificates</em>,
for communication between Kafka clients and Kafka brokers, and inter-cluster communication.</p>
</div>
<div class="paragraph">
<p>Strimzi uses <em>Secrets</em> to store the certificates and private keys required for mTLS in PEM and PKCS #12 format.</p>
</div>
<div class="paragraph">
<p>A TLS CA (certificate authority) issues certificates to authenticate the identity of a component.
Strimzi verifies the certificates for the components against the CA certificate.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Strimzi components are verified against the <em>cluster CA</em></p>
</li>
<li>
<p>Kafka clients are verified against the <em>clients CA</em></p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="security-configuration-authentication_str"><a class="link" href="#security-configuration-authentication_str">8.2. Authentication</a></h3>
<div class="paragraph">
<p>Kafka listeners use authentication to ensure a secure client connection to the Kafka cluster.</p>
</div>
<div class="paragraph">
<p>Supported authentication mechanisms:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>mTLS authentication (on listeners with TLS-enabled encryption)</p>
</li>
<li>
<p>SASL SCRAM-SHA-512</p>
</li>
<li>
<p>OAuth 2.0 token based authentication</p>
</li>
<li>
<p>Custom authentication</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The User Operator manages user credentials for mTLS and SCRAM authentication, but not OAuth 2.0.
For example, through the User Operator you can create a user representing a client that requires access to the Kafka cluster, and specify <code>tls</code> as the authentication type.</p>
</div>
<div class="paragraph">
<p>Using OAuth 2.0 token-based authentication, application clients can access Kafka brokers without exposing account credentials.
An authorization server handles the granting of access and inquiries about access.</p>
</div>
<div class="paragraph">
<p>Custom authentication allows for any type of Kafka-supported authentication. It can provide more flexibility, but also adds complexity.</p>
</div>
</div>
<div class="sect2">
<h3 id="security-configuration-authorization_str"><a class="link" href="#security-configuration-authorization_str">8.3. Authorization</a></h3>
<div class="paragraph">
<p>Kafka clusters use authorization to control the operations that are permitted on Kafka brokers by specific clients or users.
If applied to a Kafka cluster, authorization is enabled for all listeners used for client connection.</p>
</div>
<div class="paragraph">
<p>If a user is added to a list of <em>super users</em> in a Kafka broker configuration,
the user is allowed unlimited access to the cluster regardless of any authorization constraints implemented through authorization mechanisms.</p>
</div>
<div class="paragraph">
<p>Supported authorization mechanisms:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Simple authorization</p>
</li>
<li>
<p>OAuth 2.0 authorization (if you are using OAuth 2.0 token-based authentication)</p>
</li>
<li>
<p>Open Policy Agent (OPA) authorization</p>
</li>
<li>
<p>Custom authorization</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Simple authorization uses the`AclAuthorizer` and <code>StandardAuthorizer</code> Kafka plugins,
which are responsible for managing Access Control Lists (ACLs) that specify user access to various resources.
For custom authorization, you configure your own <code>Authorizer</code> plugin to enforce ACL rules.</p>
</div>
<div class="paragraph">
<p>OAuth 2.0 and OPA provide policy-based control from an authorization server.
Security policies and permissions used to grant access to resources on Kafka brokers are defined in the authorization server.</p>
</div>
<div class="paragraph">
<p>URLs are used to connect to the authorization server and verify that an operation requested by a client or user is allowed or denied.
Users and clients are matched against the policies created in the authorization server that permit access to perform specific actions on Kafka brokers.</p>
</div>
</div>
<div class="sect2">
<h3 id="security-configuration-fips_str"><a class="link" href="#security-configuration-fips_str">8.4. Federal Information Processing Standards (FIPS)</a></h3>
<div class="paragraph _abstract">
<p>Federal Information Processing Standards (FIPS) are a set of security standards established by the US government to ensure the confidentiality, integrity, and availability of sensitive data and information that is processed or transmitted by information systems.
The OpenJDK used in Strimzi container images automatically enables FIPS mode when running on a FIPS-enabled Kubernetes cluster.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<div class="title">Note</div>
</td>
<td class="content">
If you don&#8217;t want to use FIPS, you can disable it in the deployment configuration of the Cluster Operator using the <code>FIPS_MODE</code> environment variable.
</td>
</tr>
</table>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="metrics-overview_str"><a class="link" href="#metrics-overview_str">9. Monitoring</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Monitoring data allows you to monitor the performance and health of Strimzi.
You can configure your deployment to capture metrics data for analysis and notifications.</p>
</div>
<div class="paragraph">
<p>Metrics data is useful when investigating issues with connectivity and data delivery.
For example, metrics data can identify under-replicated partitions or the rate at which messages are consumed.
Alerting rules can provide time-critical notifications on such metrics through a specified communications channel.
Monitoring visualizations present real-time metrics data to help determine when and how to update the configuration of your deployment.
Example metrics configuration files are provided with Strimzi.</p>
</div>
<div class="paragraph">
<p>Distributed tracing complements the gathering of metrics data by providing a facility for end-to-end tracking of messages through Strimzi.</p>
</div>
<div class="paragraph">
<p>Cruise Control provides support for rebalancing of Kafka clusters, based on workload data.</p>
</div>
<div class="paragraph">
<div class="title">Metrics and monitoring tools</div>
<p>Strimzi can employ the following tools for metrics and monitoring:</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Prometheus</dt>
<dd>
<p><a href="https://github.com/prometheus" target="_blank" rel="noopener">Prometheus</a> pulls metrics from Kafka, ZooKeeper and Kafka Connect clusters. The Prometheus <strong>Alertmanager</strong> plugin handles alerts and routes them to a notification service.</p>
</dd>
<dt class="hdlist1">Kafka Exporter</dt>
<dd>
<p><a href="https://github.com/danielqsj/kafka_exporter" target="_blank" rel="noopener">Kafka Exporter</a> adds additional Prometheus metrics.</p>
</dd>
<dt class="hdlist1">Grafana</dt>
<dd>
<p><a href="https://grafana.com/" target="_blank" rel="noopener">Grafana Labs</a> provides dashboard visualizations of Prometheus metrics.</p>
</dd>
<dt class="hdlist1">OpenTelemetry</dt>
<dd>
<p><a href="https://opentelemetry.io/docs/" target="_blank" rel="noopener">OpenTelemetry documentation</a> provides distributed tracing support to track transactions between applications.</p>
</dd>
<dt class="hdlist1">Cruise Control</dt>
<dd>
<p><a href="https://github.com/linkedin/cruise-control" target="_blank" rel="noopener">Cruise Control</a> monitors data distribution and performs data rebalances across a Kafka cluster.</p>
</dd>
</dl>
</div>
<div class="sect2">
<h3 id="metrics-overview-tools-str"><a class="link" href="#metrics-overview-tools-str">9.1. Prometheus</a></h3>
<div class="paragraph">
<p>Prometheus can extract metrics data from Kafka components and the Strimzi Operators.</p>
</div>
<div class="paragraph">
<p>To use Prometheus to obtain metrics data and provide alerts, Prometheus and the Prometheus Alertmanager plugin must be deployed.
Kafka resources must also be deployed or redeployed with metrics configuration to expose the metrics data.</p>
</div>
<div class="paragraph">
<p>Prometheus scrapes the exposed metrics data for monitoring.
Alertmanager issues alerts when conditions indicate potential problems, based on pre-defined alerting rules.</p>
</div>
<div class="paragraph">
<p>Sample metrics and alerting rules configuration files are provided with Strimzi.
The sample alerting mechanism provided with Strimzi is configured to send notifications to a Slack channel.</p>
</div>
</div>
<div class="sect2">
<h3 id="metrics-overview-grafana_str"><a class="link" href="#metrics-overview-grafana_str">9.2. Grafana</a></h3>
<div class="paragraph">
<p>Grafana uses the metrics data exposed by Prometheus to present dashboard visualizations for monitoring.</p>
</div>
<div class="paragraph">
<p>A deployment of Grafana is required, with Prometheus added as a data source.
Example dashboards, supplied with Strimzi as JSON files, are imported through the Grafana interface to present monitoring data.</p>
</div>
</div>
<div class="sect2">
<h3 id="metrics-overview-exporter_str"><a class="link" href="#metrics-overview-exporter_str">9.3. Kafka Exporter</a></h3>
<div class="paragraph">
<p>Kafka Exporter is an open source project to enhance monitoring of Apache Kafka brokers and clients.
Kafka Exporter is deployed with a Kafka cluster to extract additional Prometheus metrics data from Kafka brokers related to offsets, consumer groups, consumer lag, and topics.
You can use the Grafana dashboard provided to visualize the data collected by Prometheus from Kafka Exporter.</p>
</div>
<div class="paragraph">
<p>A sample configuration file, alerting rules and Grafana dashboard for Kafka Exporter are provided with Strimzi.</p>
</div>
</div>
<div class="sect2">
<h3 id="metrics-overview-tracing_str"><a class="link" href="#metrics-overview-tracing_str">9.4. Distributed tracing</a></h3>
<div class="paragraph _abstract">
<p>Distributed tracing tracks the progress of transactions between applications in a distributed system.
In a microservices architecture, tracing tracks the progress of transactions between services.
Trace data is useful for monitoring application performance and investigating issues with target systems and end-user applications.</p>
</div>
<div class="paragraph">
<p>In Strimzi, tracing facilitates the end-to-end tracking of messages: from source systems to Kafka, and then from Kafka to target systems and applications.
Distributed tracing complements the monitoring of metrics in Grafana dashboards, as well as the component loggers.</p>
</div>
<div class="paragraph">
<p>Support for tracing is built in to the following Kafka components:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>MirrorMaker to trace messages from a source cluster to a target cluster</p>
</li>
<li>
<p>Kafka Connect to trace messages consumed and produced by Kafka Connect</p>
</li>
<li>
<p>Kafka Bridge to trace messages between Kafka and HTTP client applications</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Tracing is not supported for Kafka brokers.</p>
</div>
<div class="paragraph">
<p>You enable and configure tracing for these components through their custom resources.
You add tracing configuration using <code>spec.template</code> properties.</p>
</div>
<div class="paragraph">
<p>You enable tracing by specifying a tracing type using the <code>spec.tracing.type</code> property:</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1"><code>opentelemetry</code></dt>
<dd>
<p>Specify <code>type: opentelemetry</code> to use OpenTelemetry. By Default, OpenTelemetry uses the OTLP (OpenTelemetry Protocol) exporter and endpoint to get trace data. You can specify other tracing systems supported by OpenTelemetry, including Jaeger tracing. To do this, you change the OpenTelemetry exporter and endpoint in the tracing configuration.</p>
</dd>
</dl>
</div>
<div class="admonitionblock caution">
<table>
<tr>
<td class="icon">
<div class="title">Caution</div>
</td>
<td class="content">
Strimzi no longer supports OpenTracing.
If you were previously using OpenTracing with the <code>type: jaeger</code> option, we encourage you to transition to using OpenTelemetry instead.
</td>
</tr>
</table>
</div>
<h4 id="tracing_for_kafka_clients" class="discrete">Tracing for Kafka clients</h4>
<div class="paragraph">
<p>Client applications, such as Kafka producers and consumers, can also be set up so that transactions are monitored.
Clients are configured with a tracing profile, and a tracer is initialized for the client application to use.</p>
</div>
</div>
<div class="sect2">
<h3 id="metrics-overview-cruisecontrol_str"><a class="link" href="#metrics-overview-cruisecontrol_str">9.5. Cruise Control</a></h3>
<div class="paragraph _abstract">
<p>Cruise Control is an open source system that supports the following Kafka operations:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Monitoring cluster workload</p>
</li>
<li>
<p>Rebalancing a cluster based on predefined constraints</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The operations help with running a more balanced Kafka cluster that uses broker pods more efficiently.</p>
</div>
<div class="paragraph">
<p>A typical cluster can become unevenly loaded over time.
Partitions that handle large amounts of message traffic might not be evenly distributed across the available brokers.
To rebalance the cluster, administrators must monitor the load on brokers and manually reassign busy partitions to brokers with spare capacity.</p>
</div>
<div class="paragraph">
<p>Cruise Control automates the cluster rebalancing process.
It constructs a <em>workload model</em> of resource utilization for the cluster&#8212;&#8203;based on CPU, disk, and network load&#8212;&#8203;and generates optimization proposals (that you can approve or reject) for more balanced partition assignments.
A set of configurable optimization goals is used to calculate these proposals.</p>
</div>
<div class="paragraph">
<p>You can generate optimization proposals in specific modes.
The default <code>full</code> mode rebalances partitions across all brokers.
You can also use the <code>add-brokers</code> and <code>remove-brokers</code> modes to accommodate changes when scaling a cluster up or down.</p>
</div>
<div class="paragraph">
<p>When you approve an optimization proposal, Cruise Control applies it to your Kafka cluster.
You configure and generate optimization proposals using a <code>KafkaRebalance</code> resource.
You can configure the resource using an annotation so that optimization proposals are approved automatically or manually.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<div class="title">Note</div>
</td>
<td class="content">
Prometheus can extract Cruise Control metrics data, including data related to optimization proposals and rebalancing operations.
A sample configuration file and Grafana dashboard for Cruise Control are provided with Strimzi.
</td>
</tr>
</table>
</div>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2024-03-12 13:22:05 UTC
</div>
</div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.18.3/highlight.min.js"></script>
<script>
if (!hljs.initHighlighting.called) {
  hljs.initHighlighting.called = true
  ;[].slice.call(document.querySelectorAll('pre.highlight > code[data-lang]')).forEach(function (el) { hljs.highlightBlock(el) })
}
</script>
</body>
</html>